Using Hierarchical Skills for Optimized Task Assignment in Knowledge-Intensive Crowdsourcing
نویسندگان
چکیده
Besides the simple human intelligence tasks such as image labeling, crowdsourcing platforms propose more and more tasks that require very specific skills, especially in participative science projects. In this context, there is a need to reason about the required skills for a task and the set of available skills in the crowd, in order to increase the resulting quality. Most of the existing solutions rely on unstructured tags to model skills (vector of skills). In this paper we propose to finely model tasks and participants using a skill tree, that is a taxonomy of skills equipped with a similarity distance within skills. This model of skills enables to map participants to tasks in a way that exploits the natural hierarchy among the skills. We illustrate the effectiveness of our model and algorithms through extensive experimentation with synthetic and real data sets.
منابع مشابه
Optimization in Knowledge-Intensive Crowdsourcing
We present SmartCrowd, a framework for optimizing collaborative knowledge-intensive crowdsourcing. SmartCrowd distinguishes itself by accounting for human factors in the process of assigning tasks to workers. Human factors designate workers’ expertise in different skills, their expected minimum wage, and their availability. In SmartCrowd, we formulate task assignment as an optimization problem,...
متن کاملPerform Three Data Mining Tasks with Crowdsourcing Process
For data mining studies, because of the complexity of doing feature selection process in tasks by hand, we need to send some of labeling to the workers with crowdsourcing activities. The process of outsourcing data mining tasks to users is often handled by software systems without enough knowledge of the age or geography of the users' residence. Uncertainty about the performance of virtual user...
متن کاملContext-Aware Hierarchical Online Learning for Performance Maximization in Mobile Crowdsourcing
In mobile crowdsourcing, mobile users accomplish outsourced human intelligence tasks. Mobile crowdsourcing requires an appropriate task assignment strategy, since different workers may have different performance in terms of acceptance rate and quality. Task assignment is challenging, since a worker’s performance (i) may fluctuate, depending on both the worker’s current context and the task cont...
متن کاملSkill-Aware Task Assignment in Crowdsourcing Applications
Besides simple human intelligence tasks such as image labeling, crowdsourcing platforms propose more and more tasks that require very specific skills. In such a setting we need to model skills that are required to execute a particular job. At the same time in order to match tasks to the crowd, we have to model the expertise of the participants. We present such a skill model that relies on a tax...
متن کاملDynamic Task Allocation for Crowdsourcing Settings
We consider the problem of optimal budget allocation for crowdsourcing problems, allocating users to tasks to maximize our final confidence in the crowdsourced answers. Such an optimized worker assignment method allows us to “boost” the efficacy of any popular crowdsourcing estimation algorithm. We consider a mutual information interpretation of the crowdsourcing problem, which leads to a stoch...
متن کامل